How Portable are Controlled Languages Rules? A Comparison of Two Empirical MT Studies

This paper describes two studies on the effectiveness of Controlled Language (CL) rules for MT. Both studies investigated the language pair English-German and used corpora from the IT domain. However, they differ in terms of the MT engines employed (Systran vs. IBM WebSphere) and the evaluative methodologies used. Study A examines the effectiveness of CL rules by measuring temporal, technical and post-editing effort. Study B examines the effectiveness of rules by measuring comprehensibility. Both Study A and Study B concluded that some CL rules had a high impact for MT while other rules had a moderate, low or no impact. The results are compared in order to determine what, if any, common conclusions can be drawn. Our conclusions are that rules governing misspelling, incorrect punctuation, sentences longer than 25 words, and the use of personal pronouns with no antecedent in a sentence had a high impact on both post-editing effort and comprehensibility. Further, we found that the use of personal pronouns with antecedents in the same sentence and stand-alone demonstrative pronouns had a low impact, while the rule advocating the use of "in order to" in purposive clauses had no impact in either study. The paper also discusses contrasting results for both studies.